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Abstract — In this paper, we consider weighted nonbinary 
repeat multiple-accumulate (WNRMA) code ensembles obtained 
from the serial concatenation of a nonbinary rate-1 /n repeat code 
and the cascade of L > 1 accumulators, where each encoder 
is followed by a nonbinary random weighter. The WNRMA 
codes are assumed to be iteratively decoded using the turbo 
principle with maximum a posteriori constituent decoders. We 
derive the exact weight enumerator of nonbinary accumulators 
and subsequently give the weight enumerators for WNRMA code 
ensembles. We formally prove that the symbol-wise minimum dis- 
tance of WNRMA code ensembles asymptotically grows linearly 
with the block length when L > 3 and n > 2, and L = 2 and 
n > 3, for all powers of primes g > 3 considered, where q is the 
field size. Thus, WNRMA code ensembles are asymptotically good 
for these parameters. We also give iterative decoding thresholds, 
computed by an extrinsic information transfer chart analysis, on 
the gary symmetric channel to show the convergence properties. 
Finally, we consider the binary image of WNRMA code ensembles 
and compare the asymptotic minimum distance growth rates with 
those of binary repeat multiple-accumulate code ensembles. 



I. Introduction 

Weighted nonbinary repeat accumulate (WNRA) codes were 
introduced by Yang in [1] as the gary generalization of the 
celebrated binary repeat accumulate (RA) codes. The encoder 
consists of a rate i?i.op — 1/n nonbinary repeat code, a 
weighter, a random symbol interleaver, and an accumulator 
over a finite field GF(g) of size q. WNRA codes can be 
decoded iteratively using the turbo principle, and in [1] simu- 
lation results were presented that showed that these codes are 
superior to binary RA codes on the additive white Gaussian 
noise (AWGN) channel when the weighter is properly chosen. 
In a recent work [2], Kim et al. derived an approximate 
input-output weigfit enumerator (lOWE) for the nonbinary 
accumulator Based on that, approximate upper bounds on 
the maximum-likelihood (ML) decoding threshold of WNRA 
codes with gary orthogonal modulation and coherent detection 
over the AWGN channel were computed for different values 
of the repetition factor n and the field size q, showing that 
these codes perform close to capacity under ML decoding for 
large values of n and q. 
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In [3], Pfister showed that the minimum distance (rfmin) of 
binary repeat multiple-accumulate (RMA) codes, built from 
the concatenation of a repeat code with two or more accu- 
mulators, increases as the number of accumulators increase. 
In particular, it was shown in [3] that there exists a sequence 
of RMA codes with dr^in converging in the limit of infinitely 
many accumulators to the Gilbert- Varshamov bound (GVB). 
The stronger result that the typical d^\n converges to the GVB 
was recently proved in [4]. Also, in [5], it was conjectured 
by Pfister that the dmin of RMA codes asymptotically grows 
linearly with the block length, and that the growth rate is 
given by the threshold where the asymptotic spectral shape 
function becomes positive. More recently, it was shown in [4, 
6] that RMA code ensembles with two or more accumulators 
are indeed asymptotically good, in the sense that their dn^in 
asymptotically grows linearly with the block length. A formal 
proof was given in [4], and a method for the calculation of a 
lower bound on the growth rate coefficient was given in [6]. 

In a recent paper [7], the authors considered weighted non- 
binary repeat multiple-accumulate (WNRMA) code ensembles 
obtained from the serial concatenation of a nonbinary repeat 
code and the cascade of L > 1 accumulators, where each 
encoder is followed by a nonbinary weighter, as the gary 
generalization of binary RMA codes [3-6,8]. Building upon 
the approximate 10 WE for nonbinary accumulators [2], it was 
shown numerically in [7] that the (i,ni„ of WNRMA code 
ensembles grows linearly with the block length, and the growth 
rates were estimated. However, no formal proof was provided 
in [7]. In this paper, we address this issue. We derive an exact 
expression for the lOWE of a nonbinary accumulator which 
allows us to derive an exact closed-form expression for the av- 
erage weiglit enumerator (WE) of WNRMA code ensembles. 
We then analyze the asymptotic behavior of the average WE 
of WNRMA code ensembles, extending the asymptotic d^^in 
analysis in [4, 6] for binary RMA code ensembles to WNRMA 
code ensembles. In particular, we prove that the dmin of 
WNRMA code ensembles asymptotically grows linearly with 
the block length when L > i and n > 2, and L = 2 and 
n > 3, for all powers of primes g > 3 considered. Hence, 
WNRMA code ensembles are asymptotically good for these 
parameters. The obtained growth rates are very close to the 
GVB for practical values of q. However, for large values 
of g, the growth rate coefficient decreases with g, and the 
gap to the GVB starts to increase. Furthermore, we consider 
extrinsic information transfer (EXIT) charts [9] to analyze 
the convergence properties of WNRMA codes on the gary 
symmetric channel (QSC). Finally, we also consider the binary 
image of WNRMA codes. We give an expression for the 
average binary WE of nonbinary WNRMA code ensembles 
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Fig. 1. Encoder structure for WNRMA codes. 

and analyze its asymptotic behavior. We also compute the 
asymptotic dmin growth rates of the binary image of WNRMA 
code ensembles and compare them with those of binary RMA 
code ensembles. For given n, we show that the growth rate 
improves with the value of q for the considered values of 
q. Also, we compute ML decoding thresholds of the binary 
image of WNRMA code ensembles on the AWGN channel 
and show that these codes perform very close to capacity under 
ML decoding. 

Nonbinary codes of low rate are potentially useful in image 
watermarking appUcations. See, for instance, [10] where low- 
rate nonbinary turbo codes were proposed for this application. 

The remainder of the paper is organized as follows. In 
Section II, we describe the encoder structure of WNRMA 
codes. We also derive an exact expression for the lOWE of 
a nonbinary accumulator and a closed-form expression for 
the average WE of WNRMA code ensembles. In Section III, 
we analyze the asymptotic behavior of the average WE of 
WNRMA code ensembles and prove that its rfmin grows 
Unearly with the block length. Convergence properties under 
iterative decoding are studied in Section IV, where an EXIT 
chart analysis is performed. In Section V, we consider the 
binary image of WNRMA code ensembles and compare the 
c^min growth rates with those of binary RMA code ensembles. 
We also derive ML decoding thresholds for these ensembles. 
Finally, Section VI draws some conclusions. 

II. Encoder Structure and Weight Enumerators 

The encoder structure of WNRMA codes is depicted in 
Fig. 1. It is the serial concatenation of a rate i?rep = ^/n 
repetition code Crep, with the cascade of L > 1 identical 
rate-1, memory-one, gary accumulators Ci, I = 1,...,L, 
with generator polynomials g{D) = 1/(1 + £)) over a finite 
field GF(q'), through random interleavers ni, . . . ,-jtl- Each 
encoder is followed by a nonbinary weighter, which multiplies 
each symbol at its input by a nonzero gary symbol. For 
analysis purposes we consider random weighters (RWs). We 
denote by Co the {nK, K) outer block code obtained by 
concatenating together K successive codewords of Crep. The 
overall nominal code rate (avoiding termination) is denoted 
hy R = K/N = 1/n, where N = nK is the output 



block length. In more detail, a length-if information sequence 
Uo = (wo,i, . . . , uq^k) of gary symbols wo,i G {0, 1, . . . , q-1} 
is encoded by a gary repeat code. The output of the repeat 
code xo = (a;o,i, . . . , /■o.via) is fed to a nonbinary weighter 
which multipUes each symbol xo,i by a nonzero gary symbol. 
In [1], it was shown that a careful choice of the weighter can 
significantly improve performance. The resulting sequence is 
encoded by a chain of L nonbinary accumulators, preceded 
by interleavers tti, . . . , ttl. Furthermore, each accumulator is 
followed by a nonbinary RW. 



A. Average WEs for WNRMA Code Ensembles 

Let ^ be the ensemble-average nonbinary lOWE of the 
code ensemble C with input and output block length K and 
A'', respectively, denoting the average number of codewords 
of input Hamming weight w and output Hamming weight h 
over C. Here, by Hamming weight, we mean the number of 
nonzero symbols in a codeword. For convenience, we may 
simply speak of weight. Also, denote by — X]^=o '^w h 
the ensemble-average nonbinary WE of the code ensemble C, 
giving the average number of codewords of weight h over C. 
Throughout the paper we will simply speak of lOWE and WE, 
avoiding the term nonbinary, when the fact that they refer to 
nonbinary distributions is clear from the context. 

Benedetto et al. introduced in [11] the concept of uniform 
interleaver to obtain average WEs for concatenated code 
ensembles from the WEs of the constituent encoders. Since 
we are dealing with nonbinary codes, we need to extend 
the approach from [11] to consider vector- WEs. In particular, 
consider the ensemble of serially concatenated codes (SCCs) 
obtained by connecting two nonbinary encoders Ca and Cb 
through a uniform interleaver. The ensemble-average lOWE 
of the serially concatenated code ensemble can be written as 



^scc 



= E E 



N 



(1) 



where 



N 



Nl 



1 = ('i, ^2, • • • , Iq-i) is the weight vector with entries li giving 
the number of symbols i in a codeword x, and is the 
vector-IOWE of encoder Ca, giving the number of codewords 
of input weight w at the input of Ca and output vector-weight 
1 at the output of Ca, i.e., the codeword has l\ I's, h 2's, and 
so on. Likewise, a^^ is the vector-IOWE of encoder Cb giving 
the number of codewords of input vector-weight 1 and output 
weight h. In general, it is very difficult to compute the vector- 
IOWE of an encoder in closed-form. However, if encoder Ca 
is followed by a nonbinary RW, the following theorem holds. 

Theorem 1: Let C be the ensemble of codes over G¥{q) 
obtained by the serial concatenation of two nonbinary encoders 
Ca and Cb through a uniform interleaver. Furthermore, en- 
coder Ca is followed by a nonbinary RW. Also, denote by 
°Z^h '^S^ ^^'^ lOWE of encoder Ca and encoder Cb, 
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respectively. The ensemble-average lOWE of the ensemble C 
can be written as 



(2) 



Proof: Denote by C'^ the ensemble obtained by joining 
together encoder Ca and the RW. Using the concept of uniform 
interleaver, the ensemble-average lOWE of the ensemble C can 
be written as (see (1)) 



(3) 



-C' 



where a^^ is the average vector-IOWE of the ensemble of 
weighted codes Ca, weighted through the RW. 

The average vector-IOWE of the ensemble can be written 
as a function of the vector-IOWEs of encoder Ca and of the 
RW as 



Notice that the RW (over GF(q')) is such that the weight 
is preserved, i.e., a^Y is nonzero if and only if J2iZi = 
h- Therefore, we can rewrite (4) as 



-C' 



RW 



Notice also that the following property holds for a nonbinary 
random (uniform) weighter: 



g-l ,-1 



^RW 



Vd,d' such that ^d, = ^d[. 



1=1 



In other words, the vector-IOWE of the RW depends only on 
the weight / = Y^1Z\ h, and we can write 



9-1 



a^J = afj^ Vd such that ^ d^ = /. (5) 

i=l 

It is easy to verify that the vector-IOWE aff^ is given by 

( ' ) 



Finally, using (6), (5), (4), and the fact that 



(6) 
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w,d ~ "'w 



:i and Yl <^ = "f^h 



•:Eti h=i 



in (3), after some simple manipulations, we obtain (2), which 
completes the proof. ■ 
From Theorem 1 it follows that the ensemble-average lOWE 
of WNRMA code ensembles can be computed, when each 
constituent encoder is followed by a nonbinary RW, from the 
lOWEs of the component encoders, which are easier to com- 
pute in closed-form than the vector-IOWEs. Using Theorem 1 
and the concept of uniform interleaver, the ensemble-average 



lOWE of a WNRMA code ensemble Cwnrma can be written 
as 

JV JV 



a, 
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Z_/ Z_/ w,hi,...,hL-i,h 

hi=0 hL-i=0 



(7) 



where a^™"^"^^^ ^ is called the conditional weight enumer- 
ator (CWE) of Cwnrma- 

The evaluation of (7) requires the computation of the 
lOWEs of the constituent encoders, which is addressed below. 

B. lOWEsfor Memory-One Encoders and the Repetition Code 

An approximated expression for the lOWE of a gary accu- 
mulator was given in [2]. In this section, we derive the exact 
expression for the lOWE of a gary accumulator 

Theorem 2: The lOWE for rate-1, memory-one, q'ary con- 
volutional encoders over G¥{q) with generator polynomials 
g{D) = 1/(1 + D) and g{D) = 1 + D that are terminated 
to the zero state at the end of the trelhs and with input and 
output block length N can be given in closed form as 



LU./2J 

E 

fe— max(l,t(; — /i) 



N -h\fh 
k )\k 



l\/h-k 
l) \w-2k 

w-2k 

(8) 



for positive input weights w, where k is the number of error 



events. Also, a, 
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Proof: Consider a nonbinary encoder C with input and 
output length N. Denote by an error event a path through the 
trellis which diverges from the all-zero state at depth ti and 
merges again with the all-zero state at depth tf, where tf > ti. 
A nonzero codeword of input weight w and output weight h 
corresponds to the concatenation of k error events with a total 
input weight w and a total output weight h. Partition all the 
error events into equivalence classes based on their length (or, 
equivalently, based on their accumulated output weight). In 
particular, all error events within a specific class are required 
to have the same length. By considering only classes of events 
(i.e., we do not distinguish between error events within the 
same class), k error events with an accumulated output weight 
h can be concatenated (without overlapping) in 

'N - h\ fh - 1" 
k )\k-l, 
different ways. 

The next step is to consider all the error events within the 
same class. First, take a look at the structure of the error 
events. Notice that the first transition of an error event (the 
one diverging from the all-zero state) has always input weight 
one and output weight one, while the last transition of an error 
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lK\i ^^\wY[L ^L'ii-i/2j (N-hi\(hi-l\( hi-ki \ / -, xfc, / _ 9^^li-l-2fe^ 
CwNRMA ^ \w)y^ ili^l Z^fci^max(l,/i,_i-fe,) I fc, Afci-lA/ti_i-2fciM'/ W 

lho/2\ L''i/2J L'ii-i/2j ^ ^ 

_ \ ^ \ ^ _ . _ \ ^ =CwNRMA 

/ ^ / y ' ' ' / y w,hi,...,hL-i,ki,...,kL,h 

fei— max(l,/io — ^i) fe2— max(l,/ii — /i2) fci— max(l,/ii,_i — /i^) 



event (the one merging with the all-zero state) has always input 
weight one and output weight zero. Thus, the total input weight 
accumulated at the boundaries (first and last transition) of the 
error events is 2k, while the total output weight accumulated at 
the boundaries of the error events is k. Now, each error event 
has q — 1 possibilities for the first transition (since the edge 
from the all-zero state to the all-zero state is not allowed), 
therefore, overall, we have (g — 1)*^ possibilities. On the other 
hand, there is only a single possibility for the last transition 
(the one merging with the all-zero state). Finally, we must 
distribute the remaining input weight, w — 2/c, in the /i — fc 
remaining transitions (i.e., excluding the boundaries) of the 
error events, resulting in 



h-k 
w-2k 



possible distributions for the remaining input weight. Further- 
more, for each of the nonzero input weight transitions, we 
have q — 2 additional possibilities (we must exclude the edge 
of input weight zero and the edges merging with the all-zero 
state), resulting in [q — 2)"""^*^ possibilities in total. Thus, 
overall there are 



codewords of input weight w and output weight h resulting 
from the concatenation of k error events. The result for the 
encoder g{D) = 1/{1 + D) in (8) follows by summing over all 
possible values of k. The lOWE for the feedforward encoder 
with generator polynomial g{D) = 1 + D is obtained in a 
similar manner. ■ 

Notice that the formula in (8) generalizes the closed-form 
expression for the lOWE for rate-1, memory-one, binary 
convolutional encoders from [12] to the q?ay case. 

Theorem 3: The lOWE for the {riK, K) gary repetition 
code Co with input block length K can be given in closed 
form as 



r,C0 



K 



(9) 



Proof: The number of binary vectors of length K and 
weight w is (^), and the result follows by multiplying this 
number by w times the number of nonzero elements from 
GF(g). ■ 
Using (8) and (9) in (7), we get the expression (10) at the 
top of the page for the CWE (with «; > 0) of WNRMA code 
ensembles, where for conciseness ho = nw and = h. 



III. Asymptotic Analysis of the Minimum Distance 

With regard to (10) at the top of the page, without loss of 
generality we can write 

hi = piN^\i = l,...,L- 



w 



1, 



h = pN<', ki=jiN'^\i^l, , 

where 0<a<6i<&2<---< <c<1, 0<di<a< 
1, and < di < 6j_i < 1, i = 2, . . . , L. These inequalities 
can be derived from the binomial coefficients in the expression 
in (10) combined with the fact that for a binomial coefficient 
il), n>k>0. Also, a, /3i, . . . , /3l_i, 71, . . . , 7l, and p are 
positive constants. We must consider two cases: 1) at least one 
of the quantities w,hi, . . . , fci, . . . , k^, or h is of order 

o{N), and 2) all quantities w,hi,. . . , Hl-i, ki, . . . , kL, and h 
can be expressed as fractions of the block length N, i.e., a = 
&! = ••• = = di = ■ ■ ■ = d]^ = c = 1. The following 

lemma addresses the first case for weighted nonbinary repeat 
double-accumulate (WNRAA) code ensembles. 

Lemma 1: In the ensemble of WNRAA codes with block 
length N and n > 3, in the case where at least one 
of the quantities w, hi, ki, k2, or h is of order o{N), 
^^afrkTfe *fe2 h — ^ as AT — >■ cx) for all positive values 
of h. ' " " " 

Proof: The expression in (10) is very similar to the 
expression for the conditional support size enumerating func- 
tion of RMA code ensembles [8, Eq. (10)]. In particular, the 
binomial coefficients in (10) are identical to those of [8, Eq. 
(10)]. The only difference is that (10) has some extra terms in 
the form of powers of g — 1 and q — 2. Therefore, the proof 
of [8, Lemma 3] applies, with some modifications, also here. 

■ 

Lemma 1 can be generalized to the case of WNRMA code 
ensembles with L > 3. The proof is omitted for brevity. 
As a consequence of Lemma 1, we can assume that w, 
hi,. . . , fci, . . . , fci, and h are all linear in the block 
length: The average number of codewords of weight at most 
h, for some h, of WNRMA code ensembles is upper-bounded 
by 



AT^^+i max a: 



CwNRMA 



,kL,h 



which from Lemma 1 tends to zero as N tends to infinity if at 
least one of the quantities is of order o{N). Thus, the average 
number of codewords of sublinear weight of at most h tends 
to zero as N tends to infinity. 

We now address the second case by analyzing the asymp- 
totic spectral shape function. The asymptotic spectral shape 
function is defined as [13] 

1 

N' 



r(p) =limsup^lna[^;vj 



N- 
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where sup( ) denotes the supremum of its argument, p = ^ 
is the normalized output weight, and A'' is the code block 
length. If there exists some abscissa po > such that 
supp<p. r{p) < Vp* < po, and r{p) > for some p > po, 
then it can be shown that, with high probability, the dmin 
of most codes in the ensemble grows linearly with the block 
length N, with growth rate coefficient of at least po. On the 
other hand, if r(p) is strictly zero in the range (0, po). it cannot 
be proved directly whether the dmin grows linearly with the 
block length or not. In [4], it was shown that the asymptotic 
spectral shape function of RMA codes exhibits this behavior, 
i.e., it is zero in the range (0, po) and positive for some p > po- 
By combining the asymptotic spectral shapes with the use of 
bounding techniques, it was proved in [4, Theorem 6] that the 
dmin of RMA code ensembles indeed grows linearly with the 
block length with growth rate coefficient of at least pQ. 

We remark that in the rest of the paper, with a shght abuse 
of language, we sometimes refer to po as the exact value of the 
asymptotic growth rate coefficient. However, strictly speaking, 
Po is only a lower bound on it. 

Now, by using StirUng's approximation for the binomial co- 
efficient (^) ^ (,nM{k/n) f^j. ^ ^ ^y^^ j constaut, whcrc 
H( ) is the binary entropy function with natural logarithms, 
and the fact that w, h\,. . . , Hl-i, ki, . . . , Ul, and h can all 
be assumed to be of the same order as N (due to Lemma 1, 
generahzed to the general case), a, 
as 



CwNRMA ^.jjjj written 



a 



CwNRMA 



E 



,hL-i,h 

exp{/(a, A, 



3L-i,7u...nL,p)N + o{N)} 



fci, 



JV' 



when N — > oo, where a = is the normalized input weight, 
Pi — !ii is the normalized output weight of code C;, 7; 
and the function /(•) is given by 



/(/3o,^l,. ■ . ,^L-l,7l,. . .,jL,p) 
IH(/3o) 



n 

L 



x:H(A_i)+f^(i-A)H(-^ 

/=i ;=i V-^ ^ 



Pi 

A-i -27; 



L 

+ ln(5-l)^(7i-A-i) 



1=1 

L 



ln(,-2)^(A_,-27.) + ^^^^fc^ 



(11) 



where for conciseness we defined /3o = a and Pl = P- 
Finally, the asymptotic spectral shape function for WNRMA 
code ensembles can be written as 

= sup f{l3o,Pi,...,l3L-mi,---,lL,p)- 

o<A_i<i 

max(0,/3!_i-/3i)<7i< 
mm(ft,l-ft,A_i/2) 
1=1,. ..,L 

(12) 



Note that the objective function in (12), defined in (11), can 
be rewritten into [7, Eq. (6)], since 



;=i 



2ji 



A - 7; 



1=1 



(3i-i - li 
Pi 



1=1 
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Pi-i " 7i 



Thus, the approximate asymptotic spectral shape function 
given in [7, Eq. (7)] is indeed exact. Therefore, the growth 
rate coefficients computed in this section coincide with those 
in [7]. However, for finite block lengths, the lOWE of a 
nonbinary accumulator as given by Theorem 1 in [7] using 
the approximation for p{k) given in [7, Eq. (3)] (which is 
taken from [2]) is not exact. 

From (11) and (12) it can easily be verified that the asymp- 
totic spectral shape function of WNRMA code ensembles 
satisfies the recursive relation 

j.CwNRMA(0 = r^CwNRMA(l-l) _^ ^(^^ ^^1 

0<«<1 

where r''""""^"*!'), ^ > 0, is the asymptotic spectral shape 
function with / accumulators, r'^wNRMA(o) (p) = _|_ 
p In (g — 1)) is the asymptotic spectral shape function of a 
repeat code, and 



il){u,p)= sup 

inax(0,u— /9)<7< 
min(/9,l — /9,u/2) 

+ (1 - p)H 



-H(w) + pH ^ 



7 



(fi - in 



u — 2"f 



A-pj VP-7 

-h(7 - u) \n{q -l) + {u- 2j) \n{q - 2)] . 

(13) 

Lemma 2: The asymptotic spectral shape function of the 
WNRMA code ensemble is nonnegative, i.e., 

^CwNRMA(0(p) > 0, Vp G [0, 1]. 

Proof: We have r'^wNRMA(i) (p) > ^(q, p) + H{{))/n = 0. 
The general case can be proved by induction on Z. ■ 

To analyze the asymptotic dmin behavior of WNRMA code 
ensembles, we must solve the optimization problem in (11)- 
(12). An efficient algorithm to solve this problem is given in 
Appendix A. The numerical evaluation of (11)-(12) is shown 
in Figs. 4 and 5 for WNRAA and weighted nonbinary repeat 
triple-accumulate (WNRAAA) code ensembles, respectively, 
with n = 3 and g = 4, 8, 16, and 32. The asymptotic spectral 
shape function is zero in the range (0,po) and positive for 
some p > Po- In this case, we cannot conclude directly 
whether the dmin asymptotically grows linearly with the block 
length or not. However, we can prove the following theorem. 

Theorem 4: Define po = max{p* e [Q,{q — : 
^CwNRMA(p) = Vp < p*}. Then Vp* > 



N 



lim Pr(dmin < {po-p*)N)=0 



when L > 3 and n > 2, and L = 2 and n > 3, for all powers 
of primes q > 3. Thus, if po > and r^^^^^^ip) > Vp 
(see Lemma 2), then almost all codes in the ensemble have 
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asymptotic minimum distance growing linearly with N with 
growth rate coefficient of at least pQ. 

Proof: See Appendix B. ■ 

We can now prove the following theorem. 

Theorem 5: The typical dmin of WNRMA code ensembles 
when L > 3 and n>2, and L = 2 and n > 3, for all powers 
of primes 3 < (7 < 2 , grows linearly with the block length. 

Proof: The result follows from Theorem 4 by showing 
that po (^s defined in Theorem 4) is strictly positive when 
L > 3 and n > 2, and L = 2 and n > 3, for all powers of 
primes 3 < g < 2^^. 

It follows directly from the definition of the objective 
function in (11) that the asymptotic spectral shape function 
is nonincreasing in n, and thus po is nondecreasing in n. 

Furthermore, note that if we serially concatenate any nonbi- 
nary encoder whose dmin grows linearly with the block length 
with growth rate coefficient of at least po with a nonbinary 
accumulate code followed by a uniform weighter through a 
uniform interleaver, the resulting concatenated code ensemble 
will exhibit a dmin growing linearly with the block length with 
growth rate coefficient of at least [po/2]. This follows from 
the fact that the output weight /i of a nonbinary accumulate 
code is lower bounded by \^~\, where w is the nonbinary input 
weight. This follows directly from the binomial coefficient 
iw-2k} since it implies that h ~ k > w ~ 2k, from 

which it follows that w < h + k < h+ [w/2\, which implies 
that /i > [f ] ■ 

Thus, increasing n 01 L does not change the asymptotic 
dmin linear growth property. The final part of the proof con- 
siders the last dimension, i.e., what happens when q increases. 

By numerically solving the optimization problem in (12), 
we find that po = 0.1966 for q = 3, n ~ 3, and L — 2, and 
Po = 0.1519 for q — 3, n — 2, and L — 3. Furthermore, 
Figs. 2 and 3 show the value of po (computed numerically by 
solving the optimization problem in (12) as function of the 
field size q for q = 3 and q = 2\ 2 < I < 25, when ?i = 3 
and L = 2, and n = 2 and L = 3, respectively. From the 
figures, we observe that po is strictly positive for q < 2^^ in 
both cases, i.e., for both n — 3 and L — 2, and n — 2 and 
L = 3, which concludes the proof. ■ 

We remark that we have limited the value of q to 2^^, which 
is much higher than any value used in practice. However, from 
Figs. 2 and 3, we observe that the result of Theorem 5 will 
also hold for larger values of q. 

The exact values of pq are given in Table 1 for several values 
of the repetition factor n and the field size q for WNRAA 
codes. For comparison, we have also tabulated the asymptotic 
rfmin growth rate coefficient from the asymptotic GVB for 
nonbinary codes computed from 

R> l^^ IHg(Pmin) - Pmin log,(g - 1), if Pmin < ^ 

~ lO, otherwise 

where pniin is the normalized dmin, R is the asymptotic 
rate, and is the binary entropy function with base-g 

logarithms. We observe that the gap to the GVB decreases 
with increasing values of n for a fixed value of q. For a fixed 
value of n, the growth rate coefficient increases with increasing 
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Fig. 2. The value of po versus the field size q for q = 3 and q = 2', 
2 <l <25, when n = 3 and L = 2. 
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Fig. 3. The value of po versus the field size q for q = 3 and q = 2 , 
2 <l <25, when n = 2 and L = 3. 




Fig. 4. Asymptotic spectral shape function of WNRAA codes with n = 3. 

values of q, while the gap to the GVB stays approximately 
constant. However, as can be seen from Figs. 2 and 3, 
this behavior only holds for small values of q. In fact, the 
asymptotic growth rate coefficient increases with the field size 
q up to some value, and then it decreases again, after which 
the gap to the GVB also increases. This is also consistent with 
the behavior observed for nonbinary low-density parity-check 
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Fig. 5. Asymptotic spectral shape function of WNRAAA codes with n = 3. 

TABLE I 

Growth rate coefficient po of WNRAA codes for different 

VALUES OF THE REPETITION FACTOR n AND THE FIELD SIZE q. THE 
corresponding growth rates from the ASYMPTOTIC NONBINARY 
GVB ARE GIVEN IN THE PARENTHESES. 





9 = 4 


q = 8 


9 = 16 


q = 32 


n — 3 
n — 5 
n = 10 


0.2360 (0.2917) 
0.3820 (0.3977) 
0.5026 (0.5048) 


0.3107 (0.3730) 
0.4840 (0.4987) 
0.6192 (0.6207) 


0.3609 (0.4302) 
0.5518 (0.5664) 
0.6930 (0.6940) 


0.3912 (0.4715) 
0.5967 (0.6131) 
0.7413 (0.7421) 



TABLE II 

Growth rate coefficient po of WNRAAA codes for different 

VALUES OF THE REPETITION FACTOR n AND THE FIELD SIZE q. THE 
CORRESPONDING GROWTH RATES FROM THE AS YMPTOTIC NONBINARY 
GVB ARE GIVEN IN THE PARENTHESES. 





9 = 4 


9 = 8 


9 = 16 


9 = 32 


n — 3 
n = 5 
71 = 10 


0.2911 (0.2917) 
0.3977 (0.3977) 
0.5048 (0.5048) 


0.3725 (0.3730) 
0.4987 (0.4987) 
0.6207 (0.6207) 


0.4299 (0.4302) 
0.5664 (0.5664) 
0.6940 (0.6940) 


0.4712 (0.4715) 
0.6131 (0.6131) 
0.7421 (0.7421) 



codes in [14]. The values of po for WNRAAA code ensembles 
are given in Table II for selected values of n and q. The growth 
rate coefficients are very close to the GVB for WNRAA code 
ensembles with n = 5 and n = 10 and for WNRAAA code 
ensembles, for the considered values of q. For WNRAAA code 
ensembles with n — 5 and n — 10 the growth rates coincide 
with the GVB, for the considered values of q. 

IV. EXIT CHART Analysis 

In this section, we analyze the convergence properties of 
iterative decoding of WNRMA codes on the QSC using the 
turbo principle by means of an EXIT chart analysis [9]. 

The QSC is characterized by a single parameter p, which 
is the error probability of the channel. The QSC with error 
probability p takes a gary symbol at the input and outputs 
either the unchanged input symbol, with probability 1 — p, or 
any of the other q — 1 symbols, with equal probability, i.e., with 
probability p/ (q—l). The capacity C (in bits per channel use) 
of the QSC with error probability p, assuming that q = 2™, 
for some positive integer m, is given by [15] 

C = m-m2{p)-p\og^iq-l). (14) 

Asymptotically, the normalized capacity C/rn approaches 
1 — p as m tends to infinity, which is the capacity of a binary 
erasure channel with erasure probability p. 



TABLE III 

Convergence thresholds for WNRAA code ensembles for 

DIFFERENT VALUES OF THE REPETITION FACTOR n AND THE FIELD SIZE q 
ON THE QSC. The CORRESPONDING CAPACITY VALUES (COMPUTED 
FROM (14) WITH C/ log2(<j) = 1/n) ARE GIVEN IN THE PARENTHESES. 





9 = 4 


9 = 8 


9 = 16 


9 = 32 


n — 3 


0.228 (0.292) 


0.290 (0.373) 


0.335 (0.430) 


0.374 (0.471) 


n = 5 


0.263 (0.398) 


0.333 (0.499) 


0.382 (0.566) 


0.412 (0.613) 


n = 10 


0.306 (0.505) 


0.379 (0.621) 


0.424 (0.694) 


0.459 (0.742) 



The EXIT chart for a WNRMA code ensemble can be 
computed by properly combining the EXIT functions of the L 
constituent encoders Co, . . . , Cl-i (see Fig. 1) into a single 
EXIT function and then plot together in a two-dimensional 
chart this EXIT function and the EXIT function of encoder 
Cl- Note that each encoder is followed by a nonbinary RW. 
However, for notational simplicity, we assume that the RWs 
are included in Cq, . . . , Cl-i, i-c, when we speak about the 
EXIT function of Ci, we are referring to the EXIT function of 
encoder C; followed by a RW. Let /^[, and denote the 
extrinsic mutual information (MI) generated by decoder Cj^^ 
on input word u; at the input of encoder C; and on codeword 
x; at the output of Ci, respectively. Likewise, we define the a 
priori Mis by I^l^^ and I^'^^ . Consider the WNRMA encoder 
with encoders Co, ... , Cl-i as a single encoder and denote 
it by Co- We can compute the EXIT functions 

leM.-. = T'^" ila^.-.) and 4":^. = {Ia,'u, , p) (15) 

for encoders Co and Cl, respectively, where laxL-i ~ ^eviL 
and /^f, = . Note that does not depend on 

the channel while /^u^ does, since Cl is connected to the 
channel. 

The convergence behavior of WNRMA codes can now 
be tracked by displaying in a single plot the two EXIT 
curves in (15). The iterative decoding of WNRMA codes 
processes extrinsic information at the symbol level. Therefore, 
nonbinary EXIT chart analysis is required. To compute the 
EXIT functions in (15) we use the method proposed in [16] 
for turbo codes and serially concatenated codes, generalized 
to multiple serially concatenated codes. 

The convergence thresholds for WNRAA code ensembles 
predicted by the EXIT chart analysis (the maximum value 
of p such that a tunnel between the two EXIT curves in 
(15) is observed) are given in Table III for several values 
of the repetition factor n and the field size q on a QSC. 
For comparison purposes, we also report in Table III the 
corresponding capacity values computed from (14). From 
Table III it can be observed that for a given n the gap to 
capacity is similar for different values of q. On the other hand, 
given q, the iterative thresholds of WNRAA code ensembles 
are further away from capacity for increasing values of n. 

V. Binary Image of WNRMA Code Ensembles 

In this section, we consider the binary image of WNRMA 
code ensembles. We derive the average binary WE over the 
ensemble of binary images of WNRMA code ensembles, 
where each nonzero symbol from G¥(q) is mapped uniformly 
at random to nonzero binary vectors. We then compute the 
dniin asymptotic growth rates and upper bounds on the ML 
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thresholds over an AWGN channel and compare them with 
those of binary RMA code ensembles. 

Let C be a code of length N over GF(q), where q = 2™, 
and let be its nonbinary WE. Denote by a'^'^ the average 
binary WE over the ensemble of binary images of C, where 
each nonzero symbol from GF(g) is mapped uniformly at 
random to nonzero binary vectors of length m, giving the 
number of codewords of binary weight d. In the following, 
we will refer to this WE as the average binary WE of code C. 
Also, denote as before by p = the normalized nonbinary 
weight. Likewise, we denote hy 6 = the normahzed 
binary weight. The average binary WE of code C can be 
obtained from the nonbinary WE of the code as [17] 



a 



[SNmi 
min(Ar,[5ArmJ) 



■coeff (((l+x)™-l)\xL*'^™J 

(16) 



where cocff x*) is a shorthand notation for the coef- 
ficient of the monomial x* (the second argument) in the 
polynomial p{x) (the first argument). 

The binary asymptotic spectral shape function of the WN- 
RMA code ensemble is defined as [13] 

r 1 



lim sup 

N — >oo 



(17) 



where a"'^'"'^'^* '' denotes the average binary WE of the overall 
ensemble consisting of all possible binary images (obtained 
by mapping nonzero symbols from GF(q) to nonzero binary 
vectors of length m) of all codes in the WNRMA code 
ensemble. 

We will make use of the following corollary. 
Corollary 1 ([18], Corollary 16 with d= 1): 

1 

N' 



N 



lim In coeff (p(x)^, x^^) = lnp{x) — ^ Inx 



where p(x) is a polynomial in x and x is the smallest positive 
solution of 



ds 



s— In X 



Using (16) and Corollary 1 in (17), the binary asymptotic 
spectral shape function of the WNRMA code ensemble can 
be written as 



CwNRMA 



{5) 



— sup (r' 

^ i5<p<iniii(l,mi5) 



™*(p)-pln(2'"-l) 
+ pln((l + x)™-l) 



mS Inx) 
(18) 



where x is the smallest positive solution to the polynomial 
equation 

px(l+x)™-i =5((l+x)™-l) 



which simplifies, using the binomial theorem, to 

777,(5 \ /m — 1^ 



x^ = 0. 



TABLE IV 

Binary image growth rate coefficient 5o of WNRAA codes for 
different values of the repetition factor n and the field size q. 
The corresponding growth rates from the asymptotic binary 

gvb are given in the last column. 









'/ = i 


'/ = 


(/ = ll) 


,j = :\2 


GVB 


n 


3 


0.1323 [4,6] 


0.1496 


0.1608 


0.1675 


0.1712 


0.1740 


■ = 


5 


0.2286 [4, 6] 


0.2380 


0.2416 


0.2427 


0.2429 


0.2430 


n = 


10 


0.3133 [4] 


0.3155 


0.3159 


0.3160 


0.3160 


0.3160 



TABLE V 

Binary image growth rate coefficient i5o of WNRAAA codes for 

different values of the repetition factor n AND THE FIELD SIZE q. 

The corresponding growth rates from the asymptotic binary 
GVB are given in the last column. 







q = 2 


9 = 4 


q = 8 


q= 16 


<j = 32 


GVB 


n 


3 


0.1731 [4,6] 


0.1738 


0.1739 


0.1739 


0.1740 


0.1740 


• 


5 


0.2430 [4, 6] 


0.2430 


0.2430 


0.2430 


0.2430 


0.2430 


• = 


10 


0.3160 [4] 


0.3160 


0.3160 


0.3160 


0.3160 


0.3160 



To analyze the asymptotic behavior of the binary daun of 
WNRMA code ensembles, we must solve the optimization 
problem in (18), similarly to the nonbinary case. Notice that 
since the nonbinary dmin of WNRMA code ensembles grows 
hnearly with the block length, see Theorem 5, it follows that 
the (irnin of its binary image also grows Unearly with the 
block length. In Table IV, we give the binary dmin growth rate 
coefficient 6o of WNRAA code ensembles for several values 
of n and q. As a comparison, we also report the values for 
the binary RAA code ensemble (q = 2). It is observed that 
WNRAA code ensembles achieve higher growth rates than 
RAA code ensembles. For q = 32 the growth rates are very 
close to the GVB. The asymptotic binary dniin growth rates 
for WNRAAA and RAAA code ensembles are reported in 
Table V. 

A. Threshold Under ML Decoding 

The asymptotic spectral shape function of a code ensemble 
can also be used to derive a threshold under ML decoding. An 

upper bound on the ML decoding threshold of a code ensemble 
on the AWGN channel, due to Divsalar [19], is given by 

<L.r... [(l-e-'''^)(l-p) 



ML, threshold 



max 

0<p<l 



2p 



(19) 

where R is the code rate, r{p) is the asymptotic spectral shape 
function, Eb/No denotes the signal-to-noise ratio (SNR), 
and (i?6/A*'o)ML, threshold is the ML decoding threshold. We 
computed the upper bound on the ML decoding threshold 
in (19) numerically for the binary image of both WNRAA 
and WNRAAA code ensembles for several values of n and q. 
The results are given in Tables VI and VII, respectively. For 
comparison purposes, we also report in the table the binary- 
input AWGN Shannon Umit. All codes perform within 0.05 
dB from capacity. 

B. Convergence Thresholds Under Iterative Decoding 

In Table VIII, we report the iterative convergence thresholds 
for the binary image of WNRAA code ensembles on the 
AWGN channel for repeat factor n = 3 and n = 5. 
Unfortunately, while the ML thresholds improve with q and 
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TABLE VI 

Upper BOUNDS on the ML DlifODlXC, rilRliSllOLD Ol- THL binary IMACIL 01' WNRAA fODLS BASliD ON DIVS ALAR' S BOUND IN [19]. 

q — 2 — 4 Q — 8 (7—16 g — 32 Capacity 

n = 3 -0.437 -0.449 dB -0.453 dB -0.453 dB -0.453 dB -0.495 dB 
n = 5 -0.952 -0.953 dB -0953 dB -0.953 dB -0.953 dB -0.964 dB 
71 = 10 -1.284 -1.284 dB -1.284 dB -1.284 dB -1.284 dB -1.286 dB 



TABLE VII 

Upper BOUNDS on the ML decoding threshold of the binary image of WNRAAA codes based on Divs alar' s bound in [19]. 

q = 2 q = 4 g = 8 q = 16 q = 32 Capacity 

n = 3 -0.453 -0.453 dB -0.453 dB -0.453 dB -0.453 dB -0.495 dB 
n = 5 -0.953 -0.953 dB -0953 dB -0.953 dB -0.953 dB -0.964 dB 
n = 10 -1.284 -1.284 dB -1.284 dB -1.284 dB -1.284 dB -1.286 dB 



get closer to the capacity, the iterative convergence thresholds 
get worse with increasing values of q. We remark that if 
we remove the nonbinary weighters, the iterative decoding 
thresholds will improve, and they will be slightly better than 
those of binary RMA codes with the same repetition factor n. 

However, for the QSC, the iterative decoding thresholds will 
be the same with and without the nonbinary weighter (random 
or fixed). This can be explained by the following argument. 
Let the symbol-wise log-likelihood ratio (LLR) for the ith 
received symbol ru i = 1, . . . ,N, be defined as the length- 
{q — 1) vector 

\ \Pin\0)J' \ P{n\0) J J 

where P{ri\xi) is the probabiUty of receiving n when Xi 
is transmitted over the QSC. Under the assumption that we 
transmit the all-zero codeword, the symbol-wise LLR vectors 
will be of the form 

when we receive a zero (with probabihty I — p), or 

C^,ln(^i^*^),C^j (20) 

when Ti = j, j > 0. The probability of having such 
an LLR vector is p/{q — 1), independent of j (and of i). 
Since the nonbinary weighter will map a zero to a zero, 
the probability distribution of the LLR vector is preserved 
by the nonbinary weighter. Furthermore, due to properties of 
the rate-1 nonbinary accumulator (with no trellis termination), 
all symbol-wise LLR vectors of the form in (20) have the 
same probabiUty, independent of j, even at the input of the 
Lth nonbinary accumulator (after decoding on the _Lth nonbi- 
nary accumulator trelhs). Due to this property, the nonbinary 
weighter at stage L — 1 (after Cl-i) will also preserve the 
probability distribution of the symbol-wise LLR vector. The 
result follows by induction on the number of accumulators in 
the WNRMA code ensemble. 

Finally, we remark that the above argument does not apply 
to the binary image of WNRMA code ensembles transmitted 
over the binary-input AWGN channel, since the symmetry of 
the QSC is lost. 



TABLE VIII 

Convergence thresholds for the binary image of WNRAA code 

ENSEMBLES FOR DIFFERENT VALUES OF THE REPETITION FACTOR n AND 
THE FIELD SIZE q ON THE AWGN CHANNEL. 





q=2 


g = 4 


q = 8 


g = 16 


g = 32 


n = S 
n = 5 


1.68 dB 
2.77 dB 


1.94 dB 
3.10 dB 


2.23 dB 
3.46 dB 


2.45 dB 
3.76 dB 


2.66 dB 
4.07 dB 



VL Conclusion 

In this paper, we analyzed the symbol-wise minimum dis- 
tance properties of WNRMA code ensembles, where each en- 
coder is followed by a nonbinary random weighter. We derived 
an exact closed-form expression for the lOWE of nonbinary 
accumulators. Based on that, we derived the ensemble-average 
WE of WNRMA code ensembles and analyzed its asymptotic 
behavior. Furthermore, we formally proved that the symbol- 
wise minimum distance of WNRMA code ensembles asymp- 
totically grows linearly with the block length when L > 3 
and n > 2, and L = 2 and n > 3, for all powers of primes 
q > 3 considered. The asymptotic growth rate coefficient 
of the minimum distance of WNRAA and WNRAAA code 
ensembles for different values of the repetition factor n and the 
field size q were also computed. The asymptotic growth rates 
are very close to the GVB when q is large, but not too large. 
We also considered EXIT charts and analyzed the iterative 
convergence behavior of WNRMA code ensembles on the 
QSC. Finally, we considered the binary image of WNRMA 
code ensembles. We computed the asymptotic growth rates 
of their minimum distance and compared them with those 
of binary RMA code ensembles. It is shown that WNRMA 
code ensembles achieve higher dmin growth rates than their 
binary counterparts, and they get close to the GVB when q 
and n grow. Upper bounds on the ML decoding thresholds of 
the binary image of WNRMA code ensembles on the AWGN 
channel were also computed, and it was shown that WNRMA 
code ensembles perform very close to capacity under ML 
decoding. Unfortunately, iterative decoding is not able to 
fully exploit the performance of WNRMA code ensembles on 
the binary-input AWGN channel. In other words, WNRMA 
codes are excellent codes, but there is no efficient decoding 
algorithm to decode them close-to-ML. However, on the QSC, 
the iterative decoding performance is closer to capacity. 



Appendix A 
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Numerical Evaluation of the Asymptotic Spectral 
Shape Function in (12) 

The optimal value for the objective function in (11) can 
be either at a stationary point or at the boundary. To find 
the stationary points, we compute the partial derivatives with 
respect to /3o, /3i, • • ■ , and 71, ... , 7^. Setting the partial 
derivative with respect to do equal to zero gives the equation 



(g-l)(l-/?o)\'/" ' //3i-/3o + 



71 



(3o - 271 



1 



q-2 



(21) 



Setting the partial derivative with respect to 7;, I = 1, . . . , L, 
equal to zero results in the equation 

(g-2)2(/3,-/3,_i+707f 

= {q-l) {Pi-i - (1 - A - 7i) . 

Finally, setting the partial derivative with respect to / = 
1, . . . , L — 1, equal to zero results in the equation 



/3f(l-A-70(A+i-/3i + mi) 
(l-/3,)2(/3/-A-i+70(A-27i+i) 



q-2 



(23) 



To determine a solution to the above set of equations, we 
choose the following strategy. First treat ^0 as a free parameter. 
Then, from (21), solve for /3i as a (linear) function of 71 and 
insert the resulting expression into (22) for I = 1. The resulting 
third order equation can now be solved for 71. Using (23) (with 
I = 1), we can find ^2 as a (linear) function of 72 which we 
again can insert into (22) for ^ = 2. The resulting third order 
equation can then be solved for 72. Continuing like this, we 
can find the remaining values for /3; and 7;. 

In general, we must also consider all combinations of 
boundary conditions in a systematic way, since the optimum 
value may be at the boundary. Details are omitted for brevity. 



Appendix B 
Proof of Theorem 4 

The proof of Theorem 4 follows closely the proof of 
Theorem 6 in [8], which is inspired by the proof of Theorem 6 
(or Theorem 9)' in [4] and the asymptotic techniques devised 
in [20]. We start by proving Lemma 3 stated below. The lemma 
is proved by induction on L. 

Lemma 3: Let {hN}NeN be a sequence of integers such 
that for any arbitrary > 

lim = and lim ]^lJ}E. = q_ 

N — hoc N — *-oo ftAT 



Then, 



h=l 



o 



where L is the number of accumulators. 



'in [4], Theorems 6 and 9 are the same result. 



Proof: We prove the lemma by induction on the number 
of accumulators L. Consider first the case oi L = 1. We have 



h=i 



2hjs[ /n 

E 

k;=1 



CWNRA _ V- /^/" 



w 



hN Y^L"™] I ^ — h\ I h—l\ ( h—k \(„_-\\k—nw 
Z^fe=l l> k )\k-l)\nw-2k) ^) 



E 



\2k—nw 



h=l 

2/ijv / tT- 



^ -3 



w=l 



h=l 



where 



g{w,N) = 



(nM;)!e"™+"'-Viv(nw- 1) 



E 

k=l 



)tfe(fc_i)fe-i(^_2) 



\k—nw+w 



2k— nw 



and we have used the Stirling's approximation (^) < (^) 
and the fact that OLo " ^ 

'1(1+1)' 



with (t)\{l) = 

cxp ^-^2ir^ )■ ^Iso, note that the upper bound of 2hj\j/n in 
the summation over w is due to the binomial ( ''^o,). In 
more detail, h — k > nw — 2k, from which it follows that 
nw < h + k < h + [nw/2\ , which impUes that w < 2h/n. 
Now, it follows that 



2h^ /n 



iLjv JV/-" I 



h=l 



< 



2/ijv 



max 7V-r^l5(t«,iV)C^L'^^"' 



n l<w<2hN /n 

<^Ari-rtl+''g(i,Ar)/i;+LfJ-i 

= o ^ivi-r"/2i+''j 

for large enough N and for all > 0. Note that for large 

enough TV, iV™"!"^! dominates g{w,N)hL^j^^^~^ ^, due 
to the conditions on /ijv stated in the lemma. Now, assume 
that the statement of the lemma is true for the case of L — 1. 
We get 



h=l 



wnrma(j:.) 



2/ljN 



-'WNRMA(Z.-l) 



< 



E 

h=l k=l 
2hN 

E ^ 



C)(9-2) 



2k-w 



-WNRMACL 



Ibis 



h=l 
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where 



g'{w,N) = {wy.e^-'^Niw-l) 



E 

fc=l 



\k—w 



(u;-2fc)2'=-"'(g- 1) 



and C 



WNR,MA(;; 



denotes the WNRMA code ensemble with I 



accumulators. Note that the lower bound of [n/2^~^] in the 
summation over w is due to the fact that the output size h of 
an accumulator is at least \w/2], where w is the input size. 
This is due to the binomial {^l2k) '•^^ upper bound of 
[w/2\ in the summation over k. It follows that 



h=l 



WNRMA{Z.) 



< 



WNRMA(Z.-1) 



2/1 JV 

/ aw 



\9'{w,N)h 



X max A'' 
\^l7-^^<w<■2hN 

for large enough N and for all r/ > 0. Above, we used the 
induction hypothesis in the second inequality. Also, note that 

for large enough dominates (/'(w, A'') /ijy , 

due to the conditions on Hn stated in the lemma. ■ 
Lemma 4: Let r'^'^'^^"* (p; N) denote the JVth spectral 
shape function of the WNRMA code ensemble, defined as 



fCwNRMA (p. TV) = ilnaf 



^ Then, 



^CwNRMA < 2L ln[N + 1) _|_ ^CwNRMA _ 

Proof: The proof of the lemma relies on the function 
ip{u, p), defined in (13). In particular, the proof of the lemma 
is by induction on L, following the same arguments as in the 
proof of Lemma 5 in [4], and is therefore omitted for brevity. 

■ 

The final part of the proof of [4, Theorem 9] is also very 
general, and it can easily be extended to the case of WNRMA 
codes. In fact, the rest of the proof only relies on the following 
properties of ^p{u,p). 

1) 'tp{u,p) is continuous; 

2) tp{u, p), for fixed u, is strictly increasing in p < 1/2; 

3) , for fixed p, is decreasing in u; and 



4) lim^ 



< Vp < (g - l)/q. 



Finally, by using Lemmas 3 and 4 and the properties above. 
Theorem 4 is proved following the same arguments as in the 
proof of [4, Theorem 9]. Below, we will prove Properties 2 to 
4. Property 1 follows from the definition of ip{u, p) given in 
(13). 

A. Proof of Property 2 

To prove Property 2, we compute the partial derivative of 
p) with respect to p. Let the optimum value of 7 (as a 



function of p) when solving the optimization problem in (13) 
be denoted by 7p = 7(p) and its derivative with respect to p 

as 7p — j'{p)- Now, 



dip{.u,p) 

dp 



7p \ ^ 



7p 



In 



P-7p 



7p 

+ (1-7')H 



P V 7p 

%i^-p) + % 
l-p 
u - 2jp 



+ 



P-lp 
%{u-2p)-u + 2^p 



In 



In 



■7p 



■7p 



7p 



P-lp \ u 

7;in(<z-l)-27;in(g-2) 

Pj P V 7p 

7p .\^('^~P-'lp 



1 



u - 27p 
p-lp 



In 



7p 
P + 7p 



27p 



7p 
l-p 

M - 27p 

p-lp 



u ■ 



+ lp 



In 



p-lp 

Ip 



27p 
f In 



+ 



1 - P - 7p 

7p 

^P^^f P + lp 



p-lp J p-lp 
+ ln(g-l)-21n(g-2)]. 



u - 27p 



(24) 



Since 7^ is a solution to the optimization problem in (13), it 
follows that 7p is a solution to the equation 



In 



+ 



P-1 

1 

u — 2p 
p-1 



In 



1 -p-7 



In 



p + 7- 



27 



27 



p-1 

ln(g-l)-21n(g-2) = 
(25) 



which is obtained by taking the partial derivative with respect 
to 7 of the objective function in (13) and setting it equal to 
zero. Substituting (25) (with 7 = 7^) into (24), we get 

dxp{u,p) _^ flp 
P 



dp 



ip 



In 



P-7p 



+ 



= ln 



In 



7p 
l-p 
M-27p 

p-lp 
p(l 



In 



P \ Ip 

^- p-lp 
Ip 
P + lp- 



Ip 



l-p 

u - 27p 

P-7p 



u - 2-yp 
P-lp) 



(1 



P)(P + 7p -■"). 
where the last equahty follows from straightforward algebraic 
manipulations. Now, setting 



< 1 



(26) 



(l-p)(p + 7p-u) 

gives p > 1 — 7p/u > 1/2, since 7p < u/2 (see (13)) and 
the denominator in (26) is nonnegative (since ■^p > u — p, 
see (13)), from which it follows that, for fixed u, tp{u, p) is 
strictly increasing in p for p < 1/2. 
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B. Proof of Property 3 Substituting (30) into (29), we get 

d f %p{u, p)\ 1/1/1 \ 1 / ^ 
To prove Property 3, we compute the partial derivative of ^ I — ^ — j ~ —u)— pin[p) 

^ti2h£l with respect to u. We get , . , , 

- (l-p)ln(l-p) 

+ (l-p)ln(l-/0-7„) 



— I -ln(l - u) + pin 



d ( ■ip{u,p)\ 1 / dip{u,p) , 

[-^ ) = • +P Hp + %- u)) 

Let the optimum value of 7 (as a function of u) when solving 

the optimization problem in (13) be denoted by 7„ = 7(11) ^ n _ f ^ ~ P ~ '^'^ 

and its derivative with respect to m as 7^ = 7'(w). Now, \ 1 — P 

(31) 

dip{u, p) _ \Q^(^~^\^;y'\-^(P^'^'^\ function — ln(.T) is convex, and Jensen's inequahty gives 



VHf ^-^-^- VvMr^i^^ -ln(l-n) = -ln^p^^ + (l-p): 



V7«y \p-iu J (p+ju - u 

^ p-7„-2p7;+u7; ^^/ p + 7„-M \ < pln(^ ^ 

+ (74-l)ln(9-l) + (l-27;)ln(9-2) - (1 - p) In j 



7i — 27„ / from which it follows (by substituting the upper bound from 

ln(g - 1) + \n{q - 2) (32) into (31)) that (^^^) < 0, and the result follows. 



C. Proof of Property 4 

lu J \ }-a / .1 .1 I J 

y _ 27„\ u — 2p / p + — u\ Calculating the partial derivative of the objective function 

y _ 27„ J ill (13) with respect to 7 and setting it equal to zero results in 



+ r^ln 



, P - 7« / P - 7u . , _ , 

+ ln(g — 1) — 2 ln(g — 2)] equation in (30), which can be simplified to 

(27) - g^7^ + (-4 + uq^ - pq^ + Aq)-/^ 

+ {-Auq - v^q + + 4m - Apu + Apuq)^ (33) 

Since 7„ is a solution to the optimization problem in (13), it — — pu^q + pv^ + u^q = 0. 

follows that 7„ is a solution to (25). Substituting (25) (with „ . 

- ^ . o V / V Setting M = in (33), gives 

7 = 7„) mto (27), we get ^ ' ^ 

- 9^7'^ + (-4 - pq^ + Mh^ = 

^^("'^) =-ln( l^) ^ + ^"~" ^ with solutions 

du \ u J \ J (28) 

- ln(g - 1) + ln(g - 2) ^ : 

from which it follows that 



'4:-pq^+4q 
92 



Since 7„ is upper-bounded by u/2 and lower-bounded by 
d f ihiu ) \ 1 ^^^^ (13))> it follows that 70 = 7(11 = 0) = 0. Now, the limit 

- (l-p)ln(l-p) " / / ~ ^ 

+ (i-p-7„)in(i-p-7«) =1^^ ^rr + i^^Sni^' 



- 27„ ln('u - 27„) 

+ (p + 7«) ln(p + 7« - w) 



= In 



7„(ln((; - 1) -2\n{q - 2)) . ^.^ / p - 2m + 7^ + 7;,7i 

(29) « ™o 1 - 274 - 2u + 27„ + 27> 

In ( 2 ; ) + In ' ^ 





1 




2 






lim 


In 


J — >-0 






2 



u^o VCi* - 27m) (1 - w) 



Note that the equation in (25) can be simphfied to \q—lj \1 — 27q 



In 



Inf Uin ^g-^K^-^-^) ^ (30) V(9-l)(l-27^), 

'^V(^-27)V V(9-2)2(p + 7-«)y' ^ ^ (34) 



(q - 2)p 
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where the first and third equalities follow from I'Hopital's rule, 
the second equality follows from (28), and the fourth equality 
follows under the assumption that 1 — 2% is nonzero. We will 
now show that this is indeed the case. 

Since 70 = 0, we may write % = %u + 0{v?) (Taylor 
series expansion around u = 0). Substituting 7qU + O(u^) for 
7 in (33) and taking the hmit as u approaches zero, we get 

(_4 - ^ 4^) (-/ )2 + 4 _ 1) _ 1) w ^^^^ 

-l-pg + p + g = 

which has the solutions (when —4 — pq^ + 4^ is assumed to 
be nonzero) 



7o 



-2{1 -p){q- I) ±{q 



■p){q-l) 



A + pq^ - Aq 



from which it follows that 

p{q-2f±2{q- 



1-270 



2)^p{l-p){q-l) 



4 + pq^ - Aq 
Now, inserting this expression into (34), we get 



(36) 



lim 



In 



p{A + pq^ - Aq) 



(37) 



{q-l) (p{q-2)±2^p{l-p){q 



Setting 



p{A + pq^ - Aq) 



(«-l)(p(9-2)±2Vp(r= 
results in the equation 

p{-{q-l){q + 2) + pq^)=±2{q 



= 1 



l)^p{l-p){q-l). 

(38) 

Squaring both sides of the equality in (38), we get (after some 
re-arrangement) 
gV-2g2(g + 2)(g-l)p2 



+ ((g- 1)2(9 + 2)2+4(9 ■ 
with solutions 

'(<?- 

P= \ 



1)3) p- 4(9 -1)3=0 



(39) 



{A{q-l)/q\ 

The third solution is not valid, since we have assumed —4 — 
pq^ + Aq to be nonzero. Thus, (39) has only a single solution. 
To prove that the limit in (37) is strictly less than zero for 
p < iq — 1) / q when —4 — pq^ + Aq is nonzero, it is sufficient 
to evaluate the limit for some p < {q — l)/q such that —A — 
pq^+Aq is nonzero. In this respect, we choose p= {q—Vj/q^, 
which is strictly smaller than min ((9 — 1) / q^A{q — l)/(j2). 
Since the denominator in (36) is negative for this value of p, 
it follows from (36) (the ± will be a minus) that 

(g - 2)p ^ p(4 + pq^ - Aq) 

{q-l){l-2%) (g-l)(p(9-2)-2Vp(l-p)(g-l)) 



2x/?" 



< 



9+1 
3 



(9-2) 



2./(P^-{q-2) 



where the strict inequality follows from the fact that q^ — q + 
1> q^ -2q+l = {q- 1)^. Now, since g > 3, 



lim ^ = In 



{q - 2)p 



[q - 1)(1 - 2%) 



< 



u — >-o u 

for p={q- l)/g2. 
Finally, when —4 — pq"^ + Aq is zero, (35) reduces to 

A{q-l){p-l)%-l-pq + p + q = 

with solution 1 — 27q = 1/2, from which it follows from (34) 
that 

lim^=lnfMi^)<0 

for g > 5. Note that for g = 3 and 4, A{q -I) / q^ > {q-l) / q, 
and we can conclude that Property 4 is proved. 
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